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ABSTRACT  —  The  performance  evaluation  of  an  obstacle 
detection  and  segmentation  algorithm  for  Automated  Guided 
Vehicle  (AGV)  navigation  in  factory-like  environments  using  a 
3D  real-time  range  camera  is  the  subject  of  this  paper1.  Our 
approach  has  been  tested  successfully  on  British  safety  standard 
recommended  object  sizes  and  materials  placed  on  the  vehicle 
path.  The  segmented  (mapped)  obstacles  are  then  verified  using 
absolute  measurements  obtained  using  a  relatively  accurate  2D 
scanning  laser  rangefinder. 
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1.  INTRODUCTION 

Obstacle  detection  and  mapping  are  crucial  for  au¬ 
tonomous  indoor  driving.  This  is  especially  true  for  Au¬ 
tomated  Guided  Vehicle  (AGV)  navigation  in  factory-like 
environments  where  safety  of  personnel  and  that  of  the 
AGV  itself  is  of  utmost  importance.  This  paper  describes 
the  performance  of  an  obstacle  detection  and  segmentation 
algorithm  using  a  3D  real-time  range  camera. 

The  3D  range  image  camera  is  based  on  the  Time-Of- 
Flight  (TOF)  principle  [8]  and  is  capable  of  simultaneously 
producing  intensity  images  and  range  information  of  targets 
in  indoor  environments.  This  range  camera  is  extremely 
appealing  for  obstacle  detection  in  industrial  applications 
as  it  will  be  relatively  inexpensive  as  compared  to  similar 
sensors  and  can  deliver  range  and  intensity  images  at  a  rate 
of  30  Hz  with  an  active  range  of  7.5  m. 

Since  obstacle  detection  plays  a  basic  function  for  au¬ 
tonomous  driving,  there  has  been  much  research  on  many 
different  types  of  sensors,  such  as  sonar  [11],  color/gray 
level  cameras  [2],  FLIR  (Forward  Looking  InfraRed)  cam¬ 
eras  [10],  and  stereo  cameras  [9],  [1],  [12],  [6].  Most  of  the 
vision  approaches  are  not  applicable  to  indoor  scenes  due 
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equipment  identified  are  necessarily  the  best  available  for  the  purpose. 


to  lack  of  texture  in  the  environment.  Other  researchers 
have  proposed  LADAR  (Laser  Detection  And  Ranging) 
sensors  for  detecting  obstacles  [4],  [3],  [5].  However,  one 
dimension  LADAR  which  has  been  used  in  AGV  industry 
is  not  suitable  for  the  3D  world  of  factory  environments. 

Our  proposed  approach  to  obstacle  detection  uses  a  low 
cost,  3D  real-time  range  camera.  First,  we  calibrate  the 
camera  with  respect  to  the  AGV  so  that  we  can  convert 
the  range  values  to  3D  point  clouds  in  the  AGV  coordi¬ 
nate  frame.  Second,  we  segment  the  objects  which  have 
high  intensity  and  whose  elevation  values  are  above  the 
floor  of  the  operating  environment  on  the  AGV  path.  The 
segmented  3D  points  of  the  obstacles  are  then  projected 
and  accumulated  into  the  floor  surface-plane.  The  algorithm 
utilizes  the  intensity  and  3D  structure  of  range  data  from  the 
camera  and  does  not  rely  on  the  texture  of  the  environment. 
The  segmented  (mapped)  obstacles  are  verified  using  abso¬ 
lute  measurements  obtained  using  a  relatively  accurate  2D 
scanning  laser  rangefinder.  Our  approach  has  been  tested 
successfully  on  British  safety  standard  recommended  object 
sizes  and  materials  placed  on  the  vehicle  path.  In  this  paper, 
the  AGV  remained  stationary  as  the  measurements  were 
collected. 

The  U.S.  American  Society  of  Mechanical  Engineers 
(ASME)  B56.5  standard  [13]  was  recently  upgraded2  to 
allow  non-contact  safety  sensors  as  opposed  to  contact 
sensors  such  as  bumpers  on  AGVs.  Ideally,  the  U.S.  stan¬ 
dard  can  be  upgraded  further  similar  to  the  British  safety 
standard  requirements  [14].  The  British  safety  standard  of 
industrial  driverless  trucks/robots  requires  that  (a)  sensors 
shall  operate  at  least  over  the  full  width  of  the  vehicle 
and  load  in  every  direction  of  travel,  (b)  sensors  shall 
generate  a  signal  enabling  the  vehicle  to  be  stopped  by 
the  braking  system  under  specified  floor  condition  before 
contact  between  the  rigid  parts  of  the  vehicle  and/or  load 
and  a  person,  (c)  sensors  shall  detect  parts  of  a  persons 
body  as  close  as  possible  to  the  floor  but  at  least  the 

2not  cited  here  as  the  upgrade  was  not  published  prior  to  the  date  of 
this  paper. 
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“test  apparatus  shall  be  detected”,  (d)  the  activation  of 
such  sensors  shall  not  cause  injury  to  persons,  and  (e) 
reflective  characteristics  of  test  apparatus  for  personnel 
detection  means  which  work  without  physical  contact  shall 
be  representative  of  human  clothing.  We  anticipate  the  work 
described  in  this  paper  and  the  continuing  research  efforts 
to  lay  the  groundwork  towards  further  upgrade  of  the  U.S. 
safety  standards  for  AG  Vs  in  factory-like  environments. 

The  paper  is  structured  as  follows:  Section  2  describes  an 
obstacle  detection  and  segmentation  algorithm  using  range 
camera  images.  Section  3  provides  the  experimental  results 
when  the  proposed  algorithm  is  employed  for  detection  and 
segmentation  of  British  standard  test  apparatus.  Section  4 
concludes  the  paper  and  indicates  future  research  areas  that 
are  under  investigation. 

2.  OBSTACLE  DETECTION  AND  SEG¬ 
MENTATION 

In  this  section,  we  describe  an  algorithm  to  detect  and 
segment  obstacles  in  the  path  of  the  AGV  using  a  solid- 
state  Time-Of-Flight  (TOF)  range  camera.  The  3D  range 
camera  shown  in  Figure  1  is  a  compact,  robust  and  cost 
effective  solid  state  device  capable  of  producing  3D  im¬ 
ages  in  real-time.  The  camera  has  a  field-of-view  of  42° 
(horizontal)  x  46°  (vertical)  and  is  capable  of  producing 
range  images  of  160x124  pixels.  For  a  brief  overview  of 
the  characteristics  and  operating  principles  of  the  camera, 
see  [8].  Approximately  sized  British  standard  test  obstacles, 
shown  in  Figure  2,  were  placed  on  the  travel  path. 


Fig.  1.  The  TOF  3D  range  image  camera.  The  camera  simultaneously 
generates  intensity  images  and  range  information  of  targets  in  its  field-of- 
view  at  a  rate  of  30  Hz  with  an  active  range  of  7.5  m. 

The  obstacle  detection  and  segmentation  algorithm  com¬ 
bines  intensity  and  range  images  from  the  range  camera 
to  detect  the  obstacles  and  estimate  the  distance  to  the 
obstacles.  The  steps  of  the  algorithm  are  illustrated  for  a 
sample  image  from  the  camera: 

1)  First,  a  patch  data  with  high  intensity  values  (i.e.,  the 
intensity  value  is  greater  than  20)  in  the  front  of  the 
robot  are  used  to  fit  a  plane  for  estimating  the  floor 
surface  as  shown  in  Figure  3(a). 

2)  Second,  the  left  and  right  edges  of  3D  robot  paths  are 
projected  to  the  range  and  intensity  images  such  that 
only  obstacles  on  the  path  can  be  considered  as  shown 
in  Figure  3(b). 

3)  Third,  all  the  intensity  pixels  inside  of  the  left  and 
right  edges  are  used  to  hypothesize  the  potential 


(b) 


Fig.  2.  Experimental  setup,  (a)  and  (b)  depict  the  experimental  setups 
that  are  described  in  this  paper.  See  Section  3  for  further  details. 


obstacle.  If  the  intensity  value  of  the  pixel  is  greater 
than  half  of  the  average  of  the  intensity  in  the  image 
then  the  pixel  is  considered  as  a  potential  obstacle  as 
shown  in  Figure  3(c). 

4)  Fourth,  each  potential  obstacle  pixel  in  the  range 
image  is  used  to  find  the  distance  to  the  floor  plane 
when  the  distance  to  the  floor  is  greater  than  some 
threshold  as  shown  in  Figure  3(d).  The  threshold  is 
dependent  on  the  tr  avers  ability  of  the  robot. 

Potential  obstacles  in  the  world  model  can  be  accu¬ 
mulated  as  the  AGV  drives;  Figure  4  shows  an  obstacle 
map  representation  that  is  part  of  the  world  model.  The 
obstacles  map  is  shown  at  10  cm  grid  resolution.  Nearly 
all  the  obstacles  are  found,  but  at  the  cost  of  false  positives 
from  the  reflected  objects.  To  increase  the  accuracy  of  our 


obstacle  detection,  the  obstacles  in  the  map  and  information 
obtained  from  an  added  color  camera  may  be  temporally 
integrated.  Such  integration  has  proven  to  be  a  very  useful 
cue  for  obstacle  detection  [7]. 

3.  EXPERIMENTAL  SETUP  AND  RESULTS 

The  experiments  were  conducted  under  two  scenarios  as 
stated  within  the  British  Standard: 

1)  A  test  apparatus  with  a  diameter  of  200  mm  and  a 
length  of  600  mm  placed  at  right  angles  on  the  path  of  the 
AGV.  The  actuating  force  on  this  test  apparatus  shall  not 
exceed  750  N. 

2)  A  test  apparatus  with  a  diameter  of  70  mm  and  a 
height  of  400  mm  set  vertically  within  the  path  of  the  AGV. 
The  actuating  force  on  this  test  apparatus  shall  not  exceed 
250  N. 


Figures  2(a)  and  (b)  show  the  experimental  setup  for  the 
two  aforementioned  scenarios.  The  center  of  the  camera 
lens  was  centered  approximately  horizontal  and  vertical 
on  the  apparatus  for  all  measurements.  The  scanning  laser 
rangefinder  was  offset  from  the  camera  by  0  mm  vertically, 
250  mm  horizontally,  and  to  the  left  of  the  camera  as 
viewed  from  the  camera  to  the  test  apparatus.  The  range 
camera  was  used  to  detect  known  test  apparatus  mounted 
on  a  stand  and  moved  to  different  locations  with  respect  to 
the  camera. 

The  obstacle  detection  and  segmentation  algorithm  was 
tested  on  the  British  standard  test  apparatus  as  described 
in  [14],  and  was  evaluated  against  ground  truth.  A  single- 
line  scanning  laser  rangefinder,  shown  in  Figure  5,  mounted 
beside  the  range  camera,  was  used  to  simultaneously  verify 
the  distance  to  the  test  apparatus  for  each  data  set  and 
served  as  ground  truth.  The  rangefinder  produces  401  data 
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Fig.  4.  Obstacle  map. 


Fig.  5.  Experimental  setup  of  the  AGV,  the  scanning  laser  rangefinder, 
and  the  range  camera. 


points  over  a  100°  semi-circular  region  in  front  of  the  robot. 

The  obstacle  detection  and  segmentation  algorithm  was 
tested  on  British  standard  test  apparatus  which  were  placed 
in  0.5  meter  to  7.5  m  distances  to  the  sensor.  Table  1  shows 
the  performance  of  the  range  camera  for  measuring  the 
distance  to  the  test  apparatus  placed  at  several  distances 
from  the  range  camera.  As  can  be  seen,  the  accuracy  (mean) 
of  the  range  decreases  as  the  distance  of  the  apparatus 
placed  in  front  of  the  range  camera  is  increased. 

In  Figure  6,  the  test  apparatus  was  placed  at  a  distance 
of  2.5  m  from  the  range  camera.  Each  object  in  the 
test  apparatus  was  clearly  detected  even  though  the  range 
camera  was  also  sensitive  to  the  reflectors  on  the  wall  of 
the  hallway.  The  resultant  intensity,  range,  and  segmented 
images  are  shown  in  Figures  6(a),  (b)  and  (c),  respectively. 
The  ground  truth  provided  by  the  scanning  laser  rangefinder 
is  shown  in  Figure  6(d)  and  has  been  rotated  to  show  a  top- 


down  view. 

In  Figure  7,  the  test  apparatus  is  a  mannequin  leg  placed 
on  the  floor  with  an  approximate  diameter  of  200  mm 
and  a  length  of  600  mm.  This  test  apparatus  is  more 
challenging  for  the  algorithm  because  the  entire  object  is 
close  to  the  floor.  As  can  be  seen,  the  legs  are  detected, 
but  at  the  cost  of  detecting  reflectors.  Since  some  reflectors 
(see  Figure  7(c))  are  at  a  distance  of  more  than  7.5  m, 
these  are  modulated  by  the  non-ambiguity  distance  range 
of  the  camera.  This  deficiency  can  be  eliminated  by  using 
two  different  modulation  frequencies  (such  as  10  MHz  and 
20  MHz)  where  the  detected  objects  would  be  coarsely 
represented  at  a  more  appropriate  distance.  The  control 
algorithm  can  then  intelligently  delete  them. 

4.  CONCLUSIONS  AND  FURTHER  WORK 

An  obstacle  detection  and  segmentation  algorithm  for 
Automated  Guided  Vehicle  (AGV)  navigation  in  factory¬ 
like  environments  using  a  novel  3D  range  camera  was  de¬ 
scribed  in  this  paper.  The  range  camera  is  highly  attractive 
for  obstacle  detection  in  industrial  applications  as  it  will 
be  relatively  cheap  and  can  deliver  range  and  intensity 
images  in  real-time.  The  performance  of  the  algorithm  was 
evaluated  by  comparing  it  with  ground  truth  provided  by  a 
single-line  scanning  laser  rangefinder. 

We  envisage  the  extension  of  the  work  detailed  in  this 
paper  in  the  following  areas: 

•  We  believe  that  the  range  camera  can  be  used  for  mov¬ 
ing  obstacle  detection  from  a  moving  AGV.  The  detection 
of  moving  obstacles  in  the  factory  floor  is  a  next  critical 
step  for  AGV  navigation  in  such  dynamic  environments. 
Additionally,  this  sensor  can  be  combined  with  a  color 
camera  for  detecting  and  tracking  obstacles  over  long 
distances. 

•  We  also  believe  that  the  range  camera  discussed  in 
this  paper  holds  good  potential  to  be  used  in  outdoor  envi¬ 
ronments.  Towards  this,  we  have  taken  and  analyzed  some 
outdoor  data  and  the  preliminary  results  show  good  promise 
in  using  this  sensor  for  outdoor  forest  environments.  Some 
prospective  applications  include  mapping  factory  environ¬ 
ments  (“lights-out”)  manufacturing,  and  even  for  use  in 
space  due  to  its  compactness. 
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Fig.  6.  Results  of  the  obstacle  detection  and  segmentation  algorithm  for  the  experimental  setup  shown  in  Figure  2(a).  The  resultant  intensity,  range, 
and  segmented  images  are  shown  in  (a),  (b)  and  (c),  respectively.  The  ground  truth  provided  by  the  scanning  laser  rangefinder  is  shown  in  (d)  and  has 
been  rotated  to  show  a  top-down  view. 
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Fig.  7.  Results  of  the  obstacle  detection  and  segmentation  algorithm  for  the  experimental  setup  shown  in  Figure  2(b).  The  resultant  intensity,  range, 
and  segmented  images  are  shown  in  (a),  (b)  and  (c),  respectively.  The  ground  truth  provided  by  the  scanning  laser  rangefinder  is  shown  in  (d)  and  has 
been  rotated  to  show  a  top-down  view. 
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